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ABSTRACT 

Evaluation of a French and Spanish second language 
program in grades four through seven of a rural school system has 
begun with a formative evaluation for the first three years and will 
conclude with a one-year summative evaluation. The research questions 
focus on four concerns: (1) school personnel and parent perceptions 
of the program during its first year; (2) what student achievement 
can be expected after one, two, and three years of instruction at 30 
to 45 minutes twice weekly; (3) the methodology's consistency with 
current approaches; and (4) the best commercially-produced materials. 
Four data sources were considered in the first year of evaluation: 
interviews with people involved and not involved with the program, 
student scores on teacher-made tests, classroom observation, and 
preference surveys. All of these methods are to be .sed again at 
various points during the overall evaluation process. Findings about 
the program's first year include improved listening and use of memory 
among participants, improved academic achievement and attitudes among 
learning disabled students, interest among younger and older 
children, teacher and administrator interest and support, and parent 
support. Negative effects include loss of learning time, interruption 
of classes, variable student behavior upon returning from language 
instruction, ?nd scheduling difficulties. (MSE ) 
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INTRODUCTION 

The terms "ethnographic" and "evaluation" may seem 
contradictory in their most commonly-used meanings, for we often 
think of evaluation as measuring some unit against a standard, and 
of ethnographic as setting its own standard from within the unit, 
Jarvis and Adams (1979) set forth useful guidelines for the 
evaluation of second language programs. Although the nature of 
the program being evaluated plays a major role in their 
suggestions, the evaluation process they describe represents a 
deficiency model, one intended to identify weaknesses and 
strengths measured against the ideal- The decisions to be made 
after the evaluation represent ways to bring the program into 
alignment with established criteria that are generally not derived 
from the population served. By contrast, an ethnographic 
evaluation has its roots in the persons who establish the program 
and those served by it. 
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THE SCHOOL DIVISION 1 

A school division located in a small city in southwestern 
Virginia decided that an ethnographic evaluation would provide its 
leaders with precisely tht "ormation they sought. This school 
division, which historically has made decisions in a very 
democratic fashion, wanted the findings of the formative and the 
summative evaluations tc reflect the voices of the people involved 
in the school division: students, parents, and school personnel. 

This school division is located in mostly rural southwestern 
Virginia, in a small city that has a liberal arts university and 
several small industries. Total enrollment in the school system 
is approximately 1800 students. 

THE RE-FLE S PROGRAM 

The FLES program is actually called RE-FLES, Efivitalized 
foreign Language in the Elementary Schools. It is jointly funded 
by the local school division and the Virginia Department of 
Education. The program provides for 30 to 45 minutes twice weekly 
in French or Spanish for ALL students in grades five and six in 
1984-85. The program was to expand to grade four in 1985-86, but 
*n actuality, it expanded into grades four and seven. In 1986-87, 
funding plans called for expansion of the program to grade three, 
but presently plans are being made to expand to grades three and 
eight, depending on the success of the expansion in 1985-86. 
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The classes are taught by two specialists, one in each 
language, who work cooperatively with 10 classroom teachers, 
music, physical education, and art specialists. The language 
specialists were selected primarily because of their experience 
and talent for teaching elementary- aged children and secondarily 
because of their training in Spanish and French. 

Students in the fifth and sixth grade program receive no 
grade for their work, but they do receive feedback from the 
teachers about the quality of their work. £ scial efforts are 
made by the language specialists and by the regular classroom 
teachers to integrate content from both classes. Further, content 
from specialty classes, such as art, music, and physical education 
are also integrated with the language classes. 

THE EVALUATION 

The evaluation was to be formative for the first three years 
of the project, and summative upon completion of the three-year 
pilot program. It is currently in its second year. Tliis paper 
describes the overall evaluation plan and the results of the study 
of the first year of implementation. 

The areas of concern for the first year were determined 
through a four-hour interview with one administrator and two 
teachers who created the program. During this interview, the 
principle question was "What do you want to find out?" and 
discussion followed the framework set forth in the funding 
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proposal for the program. Several research questions emerged from 
this interview. They are: 

1. What are the perceptions of school division 
personnel parents and students of the program during 
its first year? Specific attention was given to 
goals, activities, outcomes, and suggestions for 
next year. 

2. What can be expected in terms of student achievement 
after one year of instruction, 30 to 45 minutes 
twice weekly? After two years and after three years 
of instruction? 

3. Is the methodology used in the classrooms consistent 
with our best knowledge about successful strategies 
for teaching elementary- aged language learners? 

4. What are the best commercially produced materials 
available for FLES programs? 



Four data sources were considered: 1) ethnographic 
face-to-face interviews with people involved and those not 
involved with the program; 2^ student scores on teacher-made 
tests; 3) direct observation of classroom instruction; and 4) 
preference surveys. The interviews, and test scores for the first 
year would provide baseline data for subsequent years of the 
study. Preference surveys would be developed for the 3econd year 
of the progr \ based on findings from the first years 1 
interviews. Comparative analysis of test scores over three years 
would be coir dieted at the end of the program. Classroom 
observation and evaluation of commercially-produced materials is 
continuous throughout all three years of the project. All 
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instruments developed for the study were informed by data 
collected from people in the school division* 

Results avialable at this point include findings from the 
ethnographic interviews, test scores from the end of the first 
year, and several classroom observations. First, I will describe 
the interview format, the interview process, and the results. The 
intent of this open-ended interview was to obtain the RANGE of 
perceptions that exist among persons involved with and not 
involved with the program, and to attempt to establish some 
relationship between their perceptions and the actual workings of 
the program. Persons interviewed represented every major group 
in the school division: students, parents, classroom teachers, 
language teachers, specialist teache:s, principals, and 
administrators. Interviewees were selected to represent positive 
and negative opinions about the program. Fifteen open-ended, 
interviews of 17 questions each were conducted in Spring, 1985. 
Questions were designed to exhaust the conceptual space of 
informants about goals, activities, outcomes, and suggestions for 
changes in the program for the second and thj rd years of the 
project. Some questions were posed singly, e.g., "What 
suggestions do you have for expanding the program to grade four 
next year?" Other questions were posed in pairs like the 
following example: 
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1. From what you know, what 1. b. What is being 
are all the goals of the done to meet the 
FLES program? (=QC) goal of X ? 

Eleven of the interviewees were directly involved with the program 
and four were not. Interviews lasted from 20 minutes to 2.5 hours. 
All responses were written, not tape-recored by the researcher to 
allow respondants mo^e time to think of things to include in their 
answers. 

FINDINGS F ROM INTERVIEWS 

The first finding is that there seems to be a great deal of 
euphoria about this program expressed by respondants throughout 
the school division, regardless of the degree of their involvement 
with the program. Very positive statements about the goals, 
activities, and outcomes were supported by rejpondants whose chief 
recommendation for change next year was to expand the program 
downward and upward through the grades. This means that the range 
of opinion sought in the interviews lies all at one end of the 
continuum. Some respondants were purposefully selected because 
it was likely that they might have a negative voice. Even they 
supported the program. The only aspects of the prograr« that were 
treated in the slightest negative way were the logistics of 
scheduling and articulation with upper- level courses, but even 
here respondants expressed great faith that the "details will be 
ironed out. 11 
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Findings about goals. Generally, respondants demonstrate 
agreement between their perceptions of the program's goal3 and 
those set forth in documents prepared within the school division 
to secare funding. Those who demonstrate the least familiarity 
with the goal3 are those who are not yet involved with the program, 
i.e., students and parents, teachers, principals in the fourth and 
lower grades and in the seventh and higher grades. The benefits 
of securing the good will of those whose lives will be altered by 
the program have already been made clear as the program is well 
received in grades 5 and 6 where lots of "groundwork" has been 
laid. The path of the program is much rockier in grade 7, where 
less consulting has been done with people who are directly 
affected by it. One respondant stated that "someone needs to look 
at the whole thing and get a grip on it." 

The program is meeting or surpassing all axpectations. 
Students are "acquiring an ear for the language," "conversational 
French is [happening] in the halls," "parents say the kids are 
bringing the language home" and teaching their siblings. There 
is a "better understanding of the geography, culture and people" 
of the countries where Spanish and French are spoken and students 
are "more tolerant of other minorities and nationalities." 
Findings about activities. The program has made itself felt in 
extensive ways across the school division and the community 
demonstrated with the activities of the program. Even those who 
are not in the program know what happens in the classrooms. They 
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know about the "warm, nonthreatening environment" that teachers 
strive to create; they know about the songs, akits, filmstrips, 
games, and prizes that are a major part of the class activities. 
One student said, at first, "Oh, we don't do much [in French 
class]." Then she went on to enumerate seven different 
activities, all in one breath, including titles of filmstrips that 
they had seen, and all the place names were pronounced in French. 

The public relations aspect of the program has been very 
effective. Among the steps taken to publicize the program were 
lots of newspaper releases; a Christmas program for parents; a 
program for the school board; handouts for students and parents 
at the beginning of the year explaining the philosophy of the 
program and everything that would be taught (samples are available 
here); teacher visits to third grade students and teachers; 
meetings with high school language teachers; parent conferences, 
etc. 

Among those areas in which there was a range of opinion is 
the fit of the REFLES program with the rest of the curriculum. 
Respondants who said the program should fit, and does, gave 
examples of how it meshes with social studies, English language 
arts, math, and science in grades five and six as well as grades 
three and four. Some respondants were able to cite chapters in 
textbooks used at various grade levels, showing how they integrate 
with the study of French or Spanish. While some respondants who 
said it does and should fit had concerns that were of a curricular 
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nature, other respondants said that it doesn't and shouldn't fit 
with the rest of the curriculum. These people felt that the 
principle distinguishing feature of the program was the enthusiasm 
it generates for learning among students, and that it does this 
by being different in its time block, its non-grade format and its 
approach to instruction. Respondants said that it "is a learning 
experience that children slip into without being aware of it"; 
that "learning is almost like osmosis, without a struggle"; that 
students are "real excited" about the class. Students themselves 
report that they "have a lot of fun"; "they get partners and talk 
with them [in Spanish or French]." 

The other side of the coin is expressed, however, by some 
respondants who say that "those who are motivated by grades are 
having trouble doing French," in an ungraded program and one 
student said that "most students enjoy the program, but there are 
about 4 out of 20 who don't." 

Findings about outcomes. The effects, or outcomes of the program 
on students were of an academic and an attitudinal nature. 
Academic effects consisted of the "discipline of listening, 
concentrating], and pay[ing] attention to a task." The program 
helps students to see concepts by "exercising their memories and 
their listening capabilities." 

Perhaps the most exciting effects of the program on students 
are the academic and attitudinal (which precedes which?) changes 
that occur for students in "special reading groups or who are 
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Learning Disabled. These students are "doing beautifully in 

French" as they "gain self-confidence in something that is thought 

of as very hard." This is such an exciting finding that I want 

to give several examples of respondants 1 comments: 

One child was the worst child in everything; 
now this child is the best in French and good 
in everything. 

A label has beer removed from a retarded child 
since he now bits in 5th grade [language] 
classroom. 

A hearing-problem child doesn't miss a beat. 

A slow reader who compensates by listening has 
a boosted self-image. 

In this program the LD child has more 
advantages than the gifted. 

Average and below-average students are 
excited to find that they can compete with 
kids that have been showing them up all these 
years in reading and writing. 

These comments support the contention of second language 
teachers that L2-study is for everyone, nwt just the gifted, or 
the elite. Second language teachers cite the way study of another 
language "can broaden students 1 perceptions of the world"; 
"encourage non-elitist ideas"; and "teach multiculturalism. 11 
Nevertheless, the public often sees the study of second languages 
as something that only the brightast or wealthiest students do. 
Perhaps these findings can help change some of those elitist 
ideas. 
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The range of opinion on the other exterme involved only one 
child whose parents asked that he not be involved in the program, 
but that he receive extra help in reading and science instead • 

Other effects - Generally, other effects of the program are good. 
Students who are too young for the program can hardly wait to begin 
it; students who are beyond its reach feel overlooked. Teachers 
are delighted because they have some extra Planning time, even if 
they stay in the classroom during the language class. Other 
teachers and principals are taking the classes right along with 
the students. Parents have been asking for this kind of program 
for a long rime and are delighted it finally exists. Principals 
are pleased because they see students learning, as shown by the 
remarks of one principal: "Whatever benefits the youngsters will 
help me. It makes for a stronger school and makes my job easier, 
that is, to help children." 

Negative effects. There are some negative effects of the program. 
These have to do with loss of academic learning time. Language 
classes interrupt the regular classroom schedule, and often 
teachers do not have time to finish a concept they have started. 
Students point out that "teachers need more time to get certain 
subjects done and they don't get it because we have to go" and that 
teachers "can't give extra help t-o students" because students must 
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go to language class. A teacher states, "I've had to cut out X # 
and I don't lijce to do that." 

Other negative effects result from student behavior upon 
return from language class. A teacher says, "Sometimes they come 
back keyed up too much." The same teacher the * says that often 
"children come back happier because of the moving around and 
dancing" that happens in language class. 

A negative effect on principals includes the difficulty of 
scheduling, especially as the program expands into grades beyond 
sixth. A principal who "read it in the paper" when the 5th and 
6th grade program began, felt equally unconsented about the 7th 
grade program. Other well-established and successful programs 
f < lied the only elective slot in the 7th grade school day, and 
language classes were scheduled for the last period of the day 
(7th), which is an add-on period* That is, school actually ends 
after the 6th period, but language classes, pports, and clubs are 
offered during the 7th period. Transportation is not available 
to students who elect to say for the 7th period. Nevertheless, 
there are enough students who want to take Spanish and French to 
offer a class for each language. 

SAMPLE PREFERENCE ITEM 

The interviews provided a range of opinion about the program, 
but no statements can be made at this point about the 
generalizability or the strength of those opinions. In order to 
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find out if a given opinion expressed in the interview data is one 
held by many people and that they feel very strongly about that 
opinion, a Preference Rating survey will be administered in the 
Spring of the second academic year (1985-86). Items like the 
following wilx appear on the survey: 



1. Spanish and French classes should be held after school. 



1 2 3 4 5 

strongly moderately no moderately strongly 

agree agree opinion disagree disagree 



[Several items intervene] 

15. Spanish and French classes should be held after school. 



1 2 3 4 5 

strongly moderately no moderately strongly 

agree agree opinion disagree disagree 



The number of respondants will give an indication of how many 
persons hold this opinion, and the total points scored for a pair 
of items wil\ indicate h<*- Wrongly people feel on this continuum 
of opinion. 
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RESULTS OF TESTS 

At the end of the first instructional year, teacher-made 
tests were given to all 5th and 6th graders who had studied French 
and Spanish. The tests were designed to examine students on what 
had been taught, not a standardized body of knowledge prescribed 
by an outside agency. Tests were of listening, speaking, reading, 
and writing skills. Sample test items are available on 
transparency. The scores were reported by the two teachers in a 
format that made comparison between languages impossible, even if 
it had been desirable. The scores were subjected to CHI-square 
and ANOVA analysis and there were no significant differences 
between or among the groups of scores for class groups or for skill 
groups. In French, reading and writing scores were lower than 
listening and speaking scores, which may reflect the purposeful 
emphasis by the teacher on the latter skills. 

These data serve as baseline data for grades 5 and 6. The 
same tests will be given in the same way in 1985-86. At that time 
t^sts for grades 4 and 7 will also be given. By the end of the 
second instructional year, then, test scores will be available for 
comparative two-year data for grades 5 and 6, and baseline data 
for grades 4 and 7. By the end of 1986-87, three-year comparative 
data will be available for grades 5 and 6; two-year comparative 
data will be available for grades 4 and 7; and baseline data for 
grades 3 and 8 will be available. 
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RECOMMENDATIONS 

These are not the usual recommendations that an evaluator 
sets forth for compliance with a standard. Instead, these are 
recommendations suggested by the people who are served by and who 
are living the REFLES program for ways to improve their own 
program. Generally, they wanted more: more time for teachers to 
work together for articulation between grades, and between 
elementary middle and hisjh schools; more time for students to take 
Spanish and French; more languages; more collaboration between 
central office and across lines between schools; more publicity; 
more materials; more planning time. 
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